Search for: All records

Distributed cloud environments running data-intensive applications often slow down because of network congestion, uneven bandwidth, and data shuffling between nodes. Traditional host metrics such as CPU or memory do not capture these factors. Scheduling without considering network conditions causes poor placement, longer data transfers, and weaker job performance. This work presents a network-aware job scheduler that uses supervised learning to predict job completion time. The system collects real-time telemetry from all nodes, uses a trained model to estimate how long a job would take on each node, and ranks nodes to choose the best placement. The scheduler is evaluated on a geo-distributed Kubernetes cluster on the FABRIC testbed using network-intensive Spark workloads. Compared to the default Kubernetes scheduler, which uses only current resource availability, the supervised scheduler shows 34–54% higher accuracy in selecting the optimal node. The contribution is the demonstration of supervised learning for real-time, network-aware job scheduling on a multi-site cluster. 

Scientific communities are increasingly using geographically distributed computing platforms. Current methods of compute placement rely on centralized controllers such as Kubernetes to match tasks with resources. This centralized model does not work well in multi-organizational collaborations. Workflows also depend on manual configurations made for a single platform and cannot adapt to changing infrastructure. This work introduces a decentralized control plane for placing computations on distributed clusters using semantic names. Semantic names are assigned to computations so they can be matched with named Kubernetes service endpoints. This approach has two main benefits. It allows job placement to be independent of location so any cluster with enough resources can run the computation. It also supports dynamic placement without requiring knowledge of cluster locations or predefined configurations. 

This work presents the design and implementation of an Open Storage System plugin for XRootD, utilizing Named Data Networking (NDN). This represents a significant step in integrating NDN, a prominent future Internet architecture, with the established data management systems within CMS. We show that this integration enables XRootD to access data in a location transparent manner, reducing the complexity of data management and retrieval. Our approach includes the creation of the NDNc software library, which bridges the existing NDN C++ library with the high-performance NDN-DPDK data-forwarding system. This paper outlines the design of the plugin and preliminary results of data transfer tests using both internal and external 100 Gbps testbed. 

High-quality traffic measurements from realistic deployments are essential for understanding and improving new network technologies. For Named Data Networking, collecting such measurements is difficult because real-world deployments are limited. To address this problem, we created a dataset of NDN traffic traces and a toolkit for capturing, analyzing, and replaying them. The dataset is collected from real routers on the official NDN testbed and is the first large, non-synthetic NDN dataset available to the research community. This paper presents the dataset and tools, describes their properties, and shares insights useful for broader NDN research.